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Abstract 

Max-product "belief propagation" is an iterative, local, message-passing algorithm for finding the maximum a posteriori (MAP) 
assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success of the algorithm 
in many application areas such as iterative decoding, computer vision and combinatorial optimization which involve graphs with 
many cycles, theoretical results about both correctness and convergence of the algorithm are known in few cases [21], [18], [23], 

^ In this paper we consider the problem of finding the Maximum Weight Matching (MWM) in a weighted complete bipartite 

graph. We define a probability distribution on the bipartite graph whose MAP assignment corresponds to the MWM. We use the 
max-product algorithm for finding the MAP of this distribution or equivalently, the MWM on the bipartite graph. Even though the 
underlying bipartite graph has many short cycles, we find that surprisingly, the max-product algorithm always converges to the 
^ ^ correct MAP assignment as long as the MAP assignment is unique. We provide a bound on the number of iterations required by 

, the algorithm and evaluate the computational cost of the algorithm. We find that for a graph of size n, the computational cost of 

the algorithm scales as O(n^), which is the same as the computational cost of the best known algorithm. Finally, we establish the 
precise relation between the max-product algorithm and the celebrated auction algorithm proposed by Bertsekas. This suggests 
possible connections between dual algorithm and max-product algorithm for discrete optimization problems. 
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^ ; I. INTRODUCTION 

Graphical models (GM) are a powerful method for representing and manipulating joint probability distributions. They have 
found major applications in several different research communities such as artificial intelligence [15], statistics [11], error- 
(N correcting codes [7], [10], [16] and neural networks. Two central problems in probabilistic inference over graphical models 
^ ^ are those of evaluating the marginal and maximum a posteriori (MAP) probabilities, respectively. In general, calculating 
the marginal or MAP probabilities for an ensemble of random variables would require a complete specification of the joint 
. probability distribution. Further, the complexity of a brute force calculation would be exponential in the size of the ensemble. 
OO ' GMs assist in exploiting the dependency structure between the random variables, allowing for the design of efficient inference 
algorithms. 

The belief propagation (BP) and max-product algorithms [15] were proposed in order to compute, respectively, the marginal 
and MAP probabilities efficiently. Comprehensive surveys of various formulations of BP and its generalization, the junction 
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Q ' tree algorithm, can be found in [2], [23], [17]. BP-based message-passing algorithms have been very successful in the context 
• • of, for example, iterative decoding for turbo codes, computer vision and finding satisfying assignments for random k-SAT. The 
. ^ simplicity, wide scope of application and experimental success of belief propagation has attracted a lot of attention recently 
ix| [2], [10], [14], [16], [24]. 

BP (or max-product) is known to converge to the correct marginal (or MAP) probabilities on tree-like graphs [15] or graphs 
with a single loop [1], [19]. For graphical models with arbitrary underlying graphs, little is known about the correctness of BP. 
Partial progress consists of [21] where the correctness of BP for Gaussian GMs is proved, [9] where an attenuated modification 
of BP is shown to work, and [16] where the iterative turbo decoding algorithm based on BP is shown to work in the asymptotic 
regime with probabilistic guarantees. To the best of our knowledge, little theoretical progress has been made in resolving the 
question: Why does BP work on arbitrary graphs? 

Motivated by the objective of providing justification for the success of BP on arbitrary graphs, we focus on the application 
of BP to the well-known combinatorial optimization problem of finding the Maximum Weight Matching (MWM) in a bipartite 
graph, also known as the "Assignment Problem". It is standard to represent combinatorial optimization problems, like finding 
the MWM, as calculating the MAP probability on a suitably defined GM which encodes the data and constraints of the 
optimization problem. Thus, the max-product algorithm can be viewed at least as a heuristic for solving the problem. In this 
paper, we study the performance of the max-product algorithm as a method for finding the MWM on a weighted complete 
bipartite graph. 

Additionally, using the max-product algorithm for problems like finding the MWM has the potential of being an exciting 
application of BP in its own right. The assignment problem is extremely well-studied algorithmically. Attempts to find better 
MWM algorithms contributed to the development of the rich theory of network flow algorithms [8], [12]. The assignment 
problem has been studied in various contexts such as job-assignment in manufacturing systems [8], switch scheduling algorithms 
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[13] and auction algorithms [6]. We believe that the max-product algorithm can be effectively used in high-speed switch 
scheduling where the distributed nature of the algorithm and its simplicity can be very attractive. 

The main result of this paper is to show that the max-product algorithm for MWM always finds the correct solution, as 
long as the solution is unique. Our proof is purely combinatorial and depends on the graph structure. We think that this result 
may lead to further insights in understanding how BP algorithms work when applied to other optimization problems. The rest 
of the paper is organized as follows: In Section |II] we provide the setup, define the Maximum Weight Matching problem (or 
assignment problem) and describe the max-product algorithm for finding the MWM. Section states and proves the main 
result of this paper Section HV-AI presents a simplification of the max-product algorithm and evaluates its computational cost. 
Section [V] discusses relation between the max-product algorithm and the celebrate auction algorithm proposed by Bertsekas. 
The auction algorithm essentially solves the dual of LP relaxation for matching problem. Our result suggests possibility of 
deeper connection between max-product and dual algorithm for optimization problems. Finally, we discuss some implications 
of our results in Section [VT] 

II. SETUP AND PROBLEM STATEMENT 

In this section, we first define the problem of finding the MWM in a weighted complete bipartite graph and then describe 
the max-product BP algorithm for solving it. 

A. MAXIMUM WEIGHT MATCHING 

Consider an undirected weighted complete bipartite graph Kn^n — (14: E), where Vi = {ai, . . . , a„}, V2 — {/3i, . . . , /3„} 
and (ai, (3j) £ E for 1 < i, j < n. Let each edge (a^, (3j) have weight Wij e K. 

If TT = {7r(l), . . . ,7r(n)} is a permutation of {1, . . . ,n} then the collection of n edges {(«!, /37r(i)): • • • i {oin, f^Trfn))} is 
called a matching of „. We denote both the permutation and the corresponding matching by tt. The weight of matching 
TT, denoted by W-^, is defined as 

= ^ Wi^(i) . 

l<j:<n 

Then, the Maximum Weight Matching (MWM), vr*, is the matching such that 

TT* = argmax^ W^^ ■ 

Note 1. In this paper, we always assume that the weights are such that the MWM is unique. In particular, if the weights of 
the edges are independent, continuous random variables, then with probability 1, the MWM is unique. 

Next, we model the problem of finding MWM as finding a MAP assignment in a graphical model where the joint 
probability distribution can be completely specified in terms of the product of functions that depend on at most two variables 
(nodes). For details about GMs, we urge the reader to see [11]. Now, consider the following GM defined on Let 
Xi, . . . , Xn, Yi, . . . ,Yn be random variables corresponding to the vertices of Kn^n and taking values from {1,2,..., n}. Let 
their joint probability distribution, p (^X — {xi, . . . , a;„); Y — (j/i, . . . , i/n)), be of the form: 

p{X,Y) = -^]^-0a.ft(a^»,?/i) J|'Aa,(a:z)0ft(y»), (1) 

where the pairwise compatibility functions, V' ('; are defined as 

{0 r = i and s ^ i 
r ^ j and s = i 
1 Otherwise 

the potentials at the nodes, (/) (•), are defined as 

./)„.(r) = 6^"-, 4>p^{r) ^ e^^^ , V 1< z,j,r,s < n, 

and Z is the normaUzation constant. We note that the pair-wise potential essentially ensures that the following two constraints 
are satisfied for any (X, Y) with positive probability: (a) If node ai is matched to node j3j (i.e Xi = j), then node /3j must be 
match to node ai (i.e. Yj = i). (b) If node ai is not matched to Pj (i.e. Xi ^ j), then node f3j must not be matched to node 
ai (i.e. Yj ^ i). These two constraints encode that the support of the above defined probability distribution is on matchings 
only. 

Claim 1: For the GM as defined above, the joint density p {X = (xi, . . . , Y — (yi, . . . , is nonzero if and only 

if -KoX^) = {(ai,l3xi)-, {a2,Px2), ("„,/?a,„)} and 7r^(F) = {{ay,, Pi), {ay^,P2),-- ■ ,(ay„,/3n)} are both matchings and 
TTa{X) — 7rfj{Y). Further, when nonzero, they are equal to ■l-e^^'"'"^. 
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When, p{X,Y) > 0, then the product of (/>.{■)'& essentially make the probability monotone function of the summation of edge 
weights as part of the corresponding matching. Formally, we state the following claim. 
Claim 2: Let {X ,Y ) be such that 

(X* ,Y*) = arg max{p (X, F) } . 

Then, the corresponding TTa{X ) — -KpiY ) is the MWM in Kn,n- 

Claim |2] implies that finding the MWM is equivalent to finding the maximum a posteriori (MAP) assignment on the GM 
defined above. Thus, the standard max-product algorithm can be used as an iterative strategy for finding the MWM. In fact 
we show that this strategy yields the correct answer Before proceeding further, we provide an example of the above defined 
GM for the ease of readability. 

Example 1: Consider a complete bipartite graph with n = 2. The random variables Xi,i = 1.2 corresponds to the index 
of P node to which ai is connected under the GM. Similarly, the random variable Yi^i ~ 1,2 correspond to the index of a 
node to which (3i is connected. For example, Xi — 1 means that ai is connected to (3i. The pair-wise potential function i/;.. 
encodes matching constraints. For example, (Xi, X2; Yi, I2) = (12; 12) corresponds to the matching where ai is connected 
to j3i and a2 is connected to (32- This is encoded (and allowed) by ip..: in this example, '4}aii3^{Xi,Y2) — ipaiP^i^i'^) = 1, 
etc. On the other hand, (Xi, X2; Yi, I2) = (12; 21) is not a matching as ai connects to (3i while /3i connects to a2- This 
is imposed by the following: tpaiPiiXijYi) = -001/31(1,2) = 0. We suggest the reader to go through this example in further 
detail by him/herself to get famihar with the above defined GM. 

B. MAX-PRODUCT ALGORITHM FOR K^^ri 

Now, we describe the max-product algorithm (and the equivalent min-sum algorithm) for the GM defined above. We need 
some definitions and notations before we can describe the max-product algorithm. Consider the following. 
Definition 1: Let D G R"^" and X,Y,Z e M"^^. Then the operations *, are defined as follows: 

D * X = Z Zi = maxdyXj, Wi, (2) 
i 

XQY^Z^z^=x,y„ Vz. (3) 

ForXi,...,X™ eM"^i, 

m 

Qx, = XiQX2Q...QX^. (4) 

Define the compatibility matrix ^q^^ G R"^" such that its (r, s) entry is ijjaii3 {r, s), for I < i, j < n. Also, let , . G 
M"^i be the following: 

= [0„,(l),...,0„.(n)]*, = [0ft. (1),..., 0/3^ (n)]* 

where A* denotes transpose of a matrix A. 
Max-Product Algorithm. 

(1) Let M^.^^. = [tiJI,.^^. (1), m^._^^. (2), . . . , m^.^^. (n)]* G K"^^ denote the messages passed from ai to Pj in the 
iteration fc > 0, for 1 < i, j < n. Similarly, M^.^^. is the message vector passed from jSj to ai in the iteration k. 

(2) Initially fc = and set the messages as follows. Let 

M„"^^ft. = [m°^_ft (1) . . . m°^_ft (n)]*, and M^%„^ = [m^^^^il) . . . m%^^^in)]\ 

where 

m°^^ft.(r) 

"^ft^a,(^) 

(3) For k > 1, messages in iteration k are obtained from messages of iteration fc — 1 recursively as follows: 

<->/3, = ^l.ft * ((O )®*".) 

Ml^^^ = vI/,,ft*((OA^„Vift)0*ft) (7) 
1^0 



gWij if r = i 
1 otherwise 



gujj, if r = I , 

(o) 

1 otherwise 
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(4) Define the beliefs (n x 1 vectors) at nodes Ui and I <i,j <n, in iteration k as follows. 




(5) The estimatecfl MWM at the end of iteration k is tt'^, where n''{i) — argmaxi<j<„{&^. (j')}, for 1 < i < n. 

(6) Repeat (3)-(5) till tt'^ converges. 



Note 2. For computational stability, it is often recommended that messages be normalized at every iteration. However, such 
normalization does not change the output of the algorithm. Since we are only interested in theoretically analyzing the algorithm, 
we will ignore the normalization step. Also, the messages are usually all initialized to one. Although the result doesn't depend 
on the initial values, setting them as defined above makes the analysis and formulas nicer at the end. 



C. MIN-SUM ALGORITHM FOR , 



The max-product and min-sum algorithms can be seen to be equivalent by observing that the logarithm function is monotone 
and hence max^ log(ai) = log(maxi ai). In order to describe the min-sum algorithm, we need to redefine , 1 < i,j < 

n, as follows: 

= [Wil, . . . , Win]*, $/3j = [wy, . . . ,W„j]*. 

Now, the min-sum algorithm is exactly the same as max-product with the equations (|7]i and (O replaced by: 
(a) Replace (01 by the following. 



(b) Replace (|7]i by the following. 



(c) Replace (O by the following. 



(r) 





if r = z 





otherwise 




if r = i 





otherwise 



(9) 
(10) 



(11) 



(12) 



Note 3. The min-sum algorithm involves only summations and subtractions compared to max-product which involves 
multiplications and divisions. Computationally, this makes the min-sum algorithm more efficient and hence very attractive. 



III. MAIN RESULT 

Now we state and prove Theorem [U which is the main contribution of this paper Before proceeding further, we need the 
following definitions. 

Definition 2: Let e be the difference between the weights of the MWM and the second maximum weight matching; i.e. 

e = W^y* - max(WTr). 

Due to the uniqueness of the MWM, e > 0. Also, define w* = maxi,j{\wij\). 

Theorem 1: For any weighted complete bipartite graph Kn.n with unique maximum weight matching, the max-product or 
min-sum algorithm when applied to the corresponding GM as defined above, converges to the correct MAP assignment or the 
MWM within iterations. 



Note that, as defined, tt^ need not be a matching. Theorem [T] shows that for large enough k, -k^ is a matching and coiTesponds to the MWM. 
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A. PROOF OF THEOREMUI 

We first present some useful notation and definitions. Consider ai, 1 < i < n. Let T^. be the level-fc unrolled tree 
corresponding to a^, defined as follows: T^. is a weighted regular rooted tree of height k + 1 with every non-leaf having 
degree n. All nodes have labels from the set . . . , Q!„, /3i, . . . , /3„} according to the following recursive rule: (a) root has 
label (b) the n children of the root have labels /3i, . . . ,/3„; and (c) the children of each non-leaf node whose parent 
has label (or /3r) have labels /3i, . . . , /J^-i, /3r+i, . . . ,(3n (or ai, . . . , a^-i, a-r+ii ■ ■ ■ , ctn)- The edge between nodes labeled 
ai,l3j in the tree is assigned weight Wij for 1 < i, j < n. Examples of such a tree for n = 3 are shown in the Figure [T] 

Note 4. is often called the level-fc computation tree at node ai corresponding to the GM under consideration. The 
computation tree in general is constructed by replicating the pairwise compatibility functions tpai^ if, s) and potentials 
4>ai{i'),(t>Pj{s), while preserving the local connectivity of the original graph. They are constructed so that the messages 
received by node after fc iterations in the actual graph are equivalent to those that would be received by the root in the 
computation tree, if the messages are passed up along the tree from the leaves to the root. 



a, a, 




(a) (b) 



Fig. 1. When n = 3 (a) is T^^ and (b) is T^^ . 

A collection A of edges in computation tree is called a T-matching if it no two edges of A are adjacent in the tree (A is a 
matching in the computation tree) and each non-leaf nodes are endpoint of exactly one edge from A. Let (r) be the weight 
of maximum weight T-matching in T*' which uses the edge {ai, (3r) at the root. 

Now, we state two important lemmas that will lead to the proof of Theorem [T] The first lemma presents an important 
characterization of the min-sum algorithm while the second lemma relates the maximum weight T-matching of the computation 
tree and the MWM in /^„^„. 

Lemma 1: At the end of the fc*'' iteration of the min-sum algorithm, the belief at node ai of Kn n is precisely 6jj — 
[24(l)...24(n)]*. 
Lemma 2: If tt* is the MWM of graph K^^n then for k > 22H£l^ 

TT*{i) = argmax{4(r)}. 

r 

That is, for fc large enough, the maximum weight T-matching in T^. chooses the edge (a^, at the root. 

Proof: [Theorem[Tl Consider the min-sum algorithm. Let 6^. = [6^. (1), . . . , 6^. (n)]*. Recall that tt*^ = (7r'^(i)) where 
7r'^(i) = argmaxr{6j;. (r)}. Then, by Lemmas [T] and |2] for fc > ^n^^ tt^ = tt*. ■ 
Next, we present the proofs of Lemmas [T] and |2] in that order. 

Proof: [Lemma [11 It is known [20] that under the min-sum (or max-product) algorithm, the vector 6^ corresponds to the 
correct max-marginals for the root ai of the MAP assignment on the GM corresponding to T^. . The pairwise compatibility 
functions force the MAP assignment on this tree to be a T-matching. Now, each edge has two endpoints and hence its weight 
is counted twice in the weight of T-matching. 

Next consider the entry of 6^ , 6^ (j). By definition, it corresponds to the MAP assignment with the value of ai at 
the root being j. That is, {ai,(3j) edge is chosen in the tree at the root. From the above discussion, 6^ (j) must be equal to 

Proof: [Lemma 121 Assume the contrary that for some fc > ^™ , 

7r*(i) ^ argmaxt^. (r) = i, for some j. (13) 
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Then, let i = 7r*(ii) for ii ^ i. Let A be the T-matching on whose weight is t^.{i)- We will modify A and find A' whose 
weight is more than A and which connects (a^, /37r*(i)) at the root instead of (a^, /37r*(ii)), thus contradicting with (fTJI i. 

First note that the set of all edges of T^. whose projection in Kn,n belong to tt* is a T-matching which we denote by 11*. 
Now consider paths P^, ^ > in , that contain edges from 11* and A alternatively defined as follows. Let — root a^, 
io — i and Pi = (ao) be a single vertex path. Let P2 = (/?7r*(j:o): Q^Oi /^7r'(ii))i where ii is such that ao — ai is connected to 
under A. For r > 1, define P2r+i and P2r+2 recursively as follows: 

P2r + 1 — {oii^rT P'2r,Ctir)7 
^2r+2 = (/37r'(i_,0j -f2r+l, /37r*(i^+i)) 

where ai_^ is the node at level 2r to which the endpoint node /3,r*(i_r+i) of path P2r is connected to under A, and v+i is 
such that ai^ at level 2r (part of P2r+i) is connected to /3^*(,,,_|_^) under A. Note that, by definition, such paths Pi for £ < k 
exist since the tree has fc + 1 levels and can support a path of length at most 2k as defined above. 

Example 2: The Figure |2jd) provides an example of such a path. The corresponding bipartite graph has n = 3 with its 
MWM shown in figure |2f a). The Figure |2fd) shows T^^, the computation tree for node ai, till level k — 5. A path, P5, 
is highlighted with thick edges alternatively complete and bold (edges from A) and dashed (edges from II*). In the figure. 
Pi = (ai); P2 = (/3i,ai,/32); Pz (^2, /3i, ai, /32, ^2) = (a3,-P2,a2) and so on. Finally, 

P5 = (ai,/32,a2,/3i,ai, ^2, 02,^3,^3) = Ci U Q, 

where Ci = (ai, 012, /32, ai) is a cycle of length 4 (see Figure |2jc)) and Q = (ai, /32, 012, Pz, 013) is a path of length 4 (see 
Figure Hb)). 

MWM 




(d) 

Fig. 2. Consider a graph with MWM shown in (a). Projection of the path for fc = 5 as shown in (d) is decomposed to (b): path Q of length 4 and (c): 
cycle Ci of length 4. The dashed edges belong to II* while bold edges belong to A. 

Now consider the path P^ of length 2k. Its edges are alternately from A and tt*. Let us refer to the edges of A as the 
A-edges of P^.. Replacing the A-edges of P^. with their complement in P^ produces a new matching A' in T^^ ; this follows 
from the way the paths are constructed. 

Lemma 3: The weight of T-matching A' is strictly higher than that of A on tree T^. . 
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This completes the proof of Lemma |2] since Lemma |3] shows that A is not the maximum weight T-matching on T^., leading 
to a contradiction. ■ 
Now, we provide the proof of Lemma [3] 

Proof: [LemmaO It suffices to show that the total weight of the A-edges is less than the total weight of their complement 
in Pfc. Consider the projection P'^, of Pk in the graph Kn.n- P'k can be decomposed into a union of a set of simple cycles 
{Ci, C2, . . . , Cm] and at most one even length path Q of length at most 2n. Since each simple cycle has at most 2n vertices 
and the length of Pk is 2k, 

m> — = (14) 
Zn n 

Consider one of these simple cycles, say Cs- Construct the matching tt' in as follows: (i) For a; £ Cs, select edges 

incident on ai that belong to A. Such edges exist by the property of the path Pk that contains Cs- (ii) For ai ^ Cg, connect 
it according to tt*, that is, add the edge (a;, 

Now tt' ^ TT* by construction. Since the MWM is unique, the definition of e gives us 

W,' < W^, - e. 

But, Wjr* — Wrr' is cxactly equal to the total weight of the n*-edges of Cs minus the total weight of the A-edges of Cs- Thus, 
weight of A-edges of Cs - weight of n* -edges of Cs = -(M^tt* - W^tt') < - £• (15) 

Since the path Q is of even length, either the first edge or the last edge is an A-edge. Without loss of generality, assume it is 
the last edge. Then, let 

Now consider the cycle 

Alternate edges of C are from the maximum weight matching tt*. Hence, using the same argument as above, we obtain 
weight of A-edges of Q - weight of n* -edges of Q = ^ u'j^.^jr*(jj^^J - ^ '^ij^-^' (ij^) 

l<r<l l<r<l 

< -t + 2w*. (16) 

From (fT4b-(fT6b. we obtain that for T-matchings A' and A in T^. : 

weight of A — weight of A' < -(to + l)(e) + 2k;* 

< -^e + 2w* 

n 

< 0. (17) 
This completes the proof of Lemma [3] ■ 



IV. Complexity 

In this section, we will analyze the complexity of the min-sum algorithm described in Section III-CI Theorem [T] suggests 
that the number of iterations required to find MWM is O ^^^^ Now, in each iteration of Min-Sum algorithm each node 
sends a vector of size n (i.e. n numbers) to each of the n nodes in the other partition. Thus, total number of messages 
exchanged in each iteration are O(n^) with each message of length n. Now, each node performs 0{n) basic computational 
operations (comparison, addition) to compute each element in a message vector of size n. That is, each node performs 0{n^) 
computational operations to compute a message vector in each iteration. Since each node sends n message vectors, the total 
cost is 0{n^) per node or 0{n'^) per iteration for all nodes. Thus, total cost for 0{nw* /e) iterations is 0{n^w* /e)- 

Thus, for fixed w* and e, the running time of algorithm scales as O(n^). The known algorithms such as Edmond-Karp's 
algorithm [8] or Auction algorithm [6] have complexity of O(n^). In what follows, we simplify the Min-Sum algorithm so that 
overall running time of the algorithm becomes 0{n^) for fixed w* and e. We make a note here that Edmond-Karp's algorithm 
is strongly polynomial (i.e. does not depend on w* and e) while Auction algorithm's complexity is 0{n^w* /e)- 



8 



A. SIMPLIFIED MIN-SUM ALGORITHM FOR X„,„ 

We first present the algorithm and show that it is exactly the same as Min-Sum algorithm. Later, we analyze the complexity 
of the algorithm. 

Simplified Min-Sum Algorithm. 

(1) Unlike Min-Sum algorithm, now each ai sends a number to jSj and vice-versa. Let the message from to Pj in iteration 
k be denoted as 

Similarly, the messages from Pj to in iteration k be denoted as 



(2) Initially fc = and set the messages as follows. 



Wi 



Similarly, 



(3) For k > 1, messages in iteration k are obtained from messages of iteration fc — 1 recursively as follows: 



Wii ~ maxm « 



(18) 



(4) The estimated MWM at the end of iteration k is tt^, where tt*'{i) = argmaxi<j<„{7fi^.^^.}, for 1 < i < n. 

(5) Repeat (3)-(4) till tt*^ converges. 



Now, we state and prove the claim that relates the above modified algorithm to the original Min-Sum algorithm. 



M; 



Lemma 4: In Min-Sum algorithm adding an equal amount to all coordinates of any message vector Mj^,^^, (similarly 
J ) at anytime does not change the resulting estimated matching tt™ for all fc, m. 
Proof: If a number is added to all coordinates of Mj^ ^^^, it is not hard to see from equation (fTTT i and structure of 
V'Qi/S ("i ■) that other message and belief vectors will change only up to an additive constant to their coordinates. Hence these 
changes do not affect 7r™(i) = argmaxi<j<„{6™. (j)}, for 1 < i < n. ■ 
Lemma 5: The algorithms Min-Sum and Simplified Min-Sum produce identical estimated matchings tt™ at the end of every 
iteration m. 

Proof: Consider the Min-Sum algorithm. In particular, consider a message vector Af^.^^ in iteration fc. First, we claim 
that all for any given fc > 0, m^._,^^ (r), r ^ i are the same. That is, for ri ^ r2 and ri, r2 ^ i. 

For fc = 0, this claim holds by definition. For fc > 1, consider the definition of {r),r ^ i. 



m'^ p (r) ^ max i{ja,p^{q,r) 

l<q<n 



(9) 



max 



til) 



(19) 



The first equality follows from definition in Min-Sum algorithm while second equality follows from property of ijjaiis ■{■,■)■ 
The equation ( fT9] l is independent of r{j^ i). This proves the desired claim. 

The above stated property of Min-Sum algorithm immediately implies that the vector Af^.^^ has only two distinct values, 
one corresponding to m'^,_^p,{i) and the other corresponding to m'^,^p,{r),r ^ i. Now subtract Ti^.^^, (r), r ^ i from 
all coordinates of . Lemma |4] guarantees the resulting matching tt™ for all m does not change. Performing the same 

modification to all message vectors yields a Modified Min-Sum algorithm with the same outcome as Min-Sum. But each 
message vector M^,^^, in this Modified Min-Sum has all coordinates equal to zero except the i*'* coordinate. Denote these 

satisfy the following recursive 



■th 



coordinates by m\ 



~k 



Now equation ( fTTT i shows these for all fc numbers fh^ 



equations; 



.0, 



maxfrn 



(20) 
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Similarly for new beliefs we have: 



bUs) = rhi^^.+Wsj (21) 



~k 



Now by adding Wij to each side of (uOt and dividing them by 2 it can be seen from ( (181 ) that numbers — - — j and 

rh^i^Pj satisfy the same recursive equations. They also satisfy the same initial conditions. As result for all i,j,k we have 

ml^_p^ ^ = 0') (22) 

and 

^ft^a. = - {i) (23) 



This shows that the estimated matching computed at nodes in Modified Min-Sum and Simphfied Min-Sum algorithms are 
exactly the same at each iteration which completes the proof of Lemma |5] ■ 

Note 5. The simplified min-sum equations can also be derived in a direct way by looking interpretation of the messages 
{™Qi^/3 i^-i.fc ^'^ '■^^ computation tree. More specifically consider the level-(fc+ 1) computation tree rooted at ai, T^^^- Also 
consider its subtree, T^J ^ , that is built by adding the edge {ai,Pj) at the root of T^+^ to graph of all descendants of f3j. 
One can show that the message m^p.^^. is equal to the difference between weight of maximum weight T-matching in ^ 
that uses the edge (aj,/3j) at the root and weight of the maximum weight T-matching in ^ that does not use that edge. 
Now a simple induction gives us the update equations ( fTSl ). 



B. COMPLEXITY OF SIMPLIFIED MIN-SUM 

The Lemma |5] and Theorem [T] immediately imply that the Simplified Min-Sum, like Min-Sum, converges after O ^-^^7-^ 
iterations. As described above, the Simplified Min-Sum algorithm requires total O(n^) messages per iteration. Thus, for fixed 
w* and e the algorithm requires total 0{ii?) messages to be exchanged. 

Now, we consider the number of computational operations done by each node in an iteration. From the description of 
Simplified Min-Sum algorithm, it may seem that each node will require to do 0{n) work for sending each message and thus 
O(n^) work overall at one node. But, we present a simple method that shows each node can compute message for all of its n 
neighbors with 0{n) computational operation (comparison, addition/subtraction). This will result in O(n^) overall computation 
per iteration. Thus, it will take O ^ " ^ computation in O ^^^^^ iterations. This will result in total complexity of O ^ " ^ 
in terms of overall messages as well as computation operations. 

Here we describe an algorithm to compute messages < j < n using received messages 'Ti^Tj';^^^ , 1 < j < n. 

This is the same algorithm that all ai,l < i < n, and f3j,l < j < n, need to employ. Now, define 

11 = argmaxi<^.<„m^;:^^^ 

12 = argmaXi<^<„j_,,^m^-_;^„^ 

Mxi = m':r^ 



Mx2 = m*' ^ 



Then, from ( fTSl ) we obtain 



rh-a^^p^^ = wiii - Mx2, 
fe 



TOai-./3, = wij-Mxi forj^ii. (24) 



We see that computing all messages m'^^^p, takes 0{n) operations. From (l24l i. it takes node ai 0{n) computations to find 
ii, 12, Mxi, Mx2, then it takes 0(1) computation to compute each of the m^^^^ , 1 < j < n. That is, total 0{n) operations 
for computing all messages m^^_^^., 1 < j <n. 

Thus, we have established that each node ck^, 1 < i < n, and < j < n, need to perform 0{n) computation to compute 
all of its messages in a given iteration. That is, the total computation cost per iteration is O(ri^). In summary. Theorem [T] 
Lemma |5] and discussion of this Section IIV-BI immediately yield the following result. 

Theorem 2: The Simplified Min-Sum algorithm finds the Maximum Weight Matching in O ^-^^^7-^ iterations with total 

computation cost of O ^ " ^ ^ and O ^ " ^ ^ total number of message exchanges. 
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V. AUCTION AND MIN-SUM 
In this section, we will first recall the auction algorithm [6] and then describe its relation to the min-sum algorithm. 

A. AUCTION ALGORITHM FOR MWM 

The Auction algorithm finds the MWM via an "auction": all ai become buyers and all [3j become objects. Let pj denote 
the price of j3j and Wij be the value of object (3j for buyer ai. The net benefit of an assignment or matching it is defined as 

n 
i=l 

The goal is to find tt* that maximizes this net benefit. It is clear that for any set of prices pi, . . . ,p„, the MWM maximizes 
the net benefit. The auction algorithm is an iterative method for finding the optimal prices and an assignment that maximizes 
the net benefit (and is therefore the MWM). 

Auction Algorithm. 

o Initialize the assignment S ~ ^, the set of unassigned buyers / = {ai, . . . , q;„}, and prices pj — for all j. 
o The algorithm runs in two phases, which are repeated until 5 is a complete matching, 
o Phase 1: Bidding. 
For all ai £ /, 

(1) Find benefit maximizing (3j. Let, 

ji — argmax {wij —Pj}, Vi — max{wy —Pj}, and Ui = max{wy —pj}. (25) 

(2) Compute the "bid" of buyer a^, denoted by b^i^pj. as follows: given a fixed positive constant S, 

ba,^l3j- = Wij, -Ui + 6. 

o Phase 2: Assignment. 
For each object (3j, 

(3) Let P{j) be the set of buyers from which Pj received a bid. If P{j) ^ 0, increase pj to the highest bid, 

Pj = max 6a.^/3,- 

(4) Remove the maximum bidder ai^ from / and add {ai^ , (3j) to S. If (offe, (3j) ^ S, k ^ ij, then put ak back in /. 

Theorem 3 ([5]): If < (5 < e/n, then the assignment S converges to the MWM in 0{nw* /e) iterations with running time 
0{n^w* / e) (where e and w* are as defined earlier). 

B. CONNECTING MIN-SUM AND AUCTION 

The similarity between equations ( |24] | and ( l25T l suggests a connection between the min-sum and auction algorithms. Next, we 
describe modifications to the min-sum and auction algorithms, called min-sum auction I and min-sum auction II, respectively. 
We will show that these versions are equivalent and derive some of their key properties. Here we consider the naive auction 
algorithm (when 5 = 0) and deal with the case (5 > in the next section. 
Min-Sum Auction I 

(1) Each ai sends a number to f3j and vice-versa. Let the messages in iteration k be denoted as rh'^,^^,, rhp,_^^, e M. 

(2) Initialize fc = and set to^.^^. = 0. 

(3) For k > 1, update messages as follows: 

"^Q.^ft = w^j - maxiw^i - TO^7_i„J, 

"^ft-*a, = max mi^^f^^, (26) 

(4) The estimated MWM at the end of iteration k is the set of edges 

tt'^' = {(a,^ ,/3i)| ij = arg max {m^,^^.} I < j < n, and m^, .^^. > m'^i } 

l<.£<.n J ' ^ ^ 

(5) Repeat (3)-(4) till tt'' is a complete matching. 



Min-Sum Auction II. 
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o Initialize the assignment 5 = and prices pj — for all j. 

o The algorithm runs in two phases, which are repeated until 5 is a complete matching, 
o Phase 1: Bidding. 
For all ai, 

(1) Find 13 j that maximizes the benefit. Let, 

ji = ai-gmax .{TOij ~ Pj}, Vi = max{wij —Pj}, and m = max{wij — Pj}. (27) 

3 j^'ii 



(2) Compute the "bid" of buyer ai, denoted by ba^^is/. 

hci^p-. = Wij^ - Ui, and fca^^/j^ = - Vi, j ^ j^. 

o Phase 2: Assignment. 
For each object (3j, 

(3) Set price pj to the highest bid, pj = maxQ. bai^f3 ■ 

(4) Reset S = $. Then, for each j add the pair {ai^,f3j) to S if ha > Pj, where ai. is a buyer attaining the 
maximum in step (3). 



Theorem 4: The algorithms min-sum auction I and II are equivalent. 

Proof: Let 6^.^^ and p^ denote the bids and prices at the end of iteration k in algorithm min-sum auction II. Now, 
identify 6^.^^ with ?ti^.^^ and p^ with . Then it is immediate that min-sum auction II becomes identical to min-sum 

auction I. This completes the proof of Theorem |4| ■ 

Next we will prove that if the min-sum auction algorithm terminates (we omit reference to I or II), it finds the correct 
maximum weight matching. As we will see, the proof uses standard arguments (see [6] for example). 

Theorem 5: Let a be the termination matching of the min-sum auction I (or II). Then it is the MWM, i.e. a — tt* . 

Proof: The proof follows by establishing that at termination, the messages of min-sum auction form the optimal solution 
for the dual of the MWM problem and a is the corresponding optimal solution to the primal, i.e. MWM. To do so, we first 
state the dual of the MWM problem 



=1 j=i 



subject to Ti+pj > Wij. (28) 

Let (r*,p*) be the optimal solution to the above stated dual problem and let tt* solve the primal MWM problem. Then, the 
standard complimentary slackness conditions are: 

rt+Pl'(i) = W,„,(^iy (29) 

Thus, {r*,p*,7T*) are the optimal dual-primal solution for the MWM problem if and only if (a) tt* is a matching, (b) {r*,p*) 
satisfy ( |28T l, and (c) the triple satisfies ( |29l ). To complete the proof we will prove the existence of r*,p* such that {r*,p*,a) 
satisfy (a), (b) and (c). 

To this end, first note that ct is a matching by the termination condition of the algorithm; thus, condition (a)is satisfied. We'll 
consider the min-sum auction II algorithm for the purpose of the proof. Suppose the algorithm terminates at some iteration k. 
Let and pj be the prices of f5j in iterations A; — 1 and k respectively. Since all (3jS are matched at the termination, from 
step (4) of the min-sum auction II, we obtain 

p'j > pY\ Vj. (30) 

At termination (iteration /c), ai is matched with iia{i) or fij is matched with Q;o.-i(j). By the definition of the min-sum auction 
II algorithm, 

p) = - max [w„-i(j)t - p\~^] . (31) 



From ([30| and (EB, we obtain that 



^^3 



'^<^-H3)3 ^ p'j - ^S''^['^<7-Hj)e -Pe] ■ (32) 



^7^3 



Define, r* — Wi^(^i^ ~Pa{i) ™^ Pj ^ Pj - Then, from ( [32b {r* ,p*) satisfy the dual feasibility, that is (|28] |. Further, by definition 
they satisfy the complimentary slackness condition ( |29] l. Thus, the triple {r* ,p*, a) satisfies (a), (b) and (c) as required. Hence, 
the algorithm min-sum auction II produces the MWM, i.e. a — tt*. ■ 
The min-sum auction II algorithm looks very similar to the auction algorithm and inherits some of its properties. However, 
it also inherits some properties of the min-sum algorithm. This causes it to behave differently from the auction algorithm. The 
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proof of convergence of auction algorithm relies on two properties of the auctioning mechanism: (a) the prices are always 
non-decreasing and (b) the number of matched objects is always non-decreasing. By design, (a) and (b) can be shown to hold 
for the auction algorithm. However, it is not clear if (a) and (b) are true for min-sum auction. In what follows, we state the 
result that prices are eventually non-decreasing in the min-sum auction algorithm; however it seems difficult to estabUsh a 
statement similar to (b) for the min-sum algorithm as of now. 

Theorem 6: If tt* is unique then in the min-sum auction II algorithm, prices eventually increase. That is, Vfc G Z+ ; 3 T > 
k s.t. yt > T; p* > p*^, l<j<n 

Proof: Proof of Theorem (|6]l is essentially based on (i) the equivalence between the min-sum auction algorithms I and 
II, and (ii) arguments very similar to the ones used in the proof of Lemma |2] , where we relate prices with the computation 
tree. ■ 

Our simulations suggests that in the absence of the condition "m^ . > J^^ " from step (4) of min-sum auction I, the 
algorithm always terminates and finds the MWM as long as it is unique. This along with Theorem |6] leads us to the following 
conjecture. 

Conjecture 1: If tt* is unique then the min-sum auction I terminates in a finite number of iterations if condition "to'; > 
rhp~^^ " is removed from step (4). 

C. RELATION TO S-RELAXATION 

In the previous section, we established a relation between the min-sum and auction (with S ~ 0) algorithms. In [6], [5] the 
author extends the auction algorithm to obtain guaranteed convergence in a finite number of iterations via a (5-relaxation for 
some (5 > 0. At termination the (5-relaxed algorithm produces a triple {r* ,p*,tt*) such that (al) tt* is a matching, (bl) {r* ,p*) 
satisfy (|28] | and (cl) the following modified complimentary slackness conditions are satisfied: 

^t+Pl'ii) < (33) 

The conditions (cl) are referred to as 5-CS conditions in [6]. This modification is reflected in the description of the auction 
algorithm where we have added 6 to each bid in step (2). We established the relation between min-sum and auction for 
5 = in the previous section. Here we make a note that for every S > 0, the similar relation holds. To see this, we 
consider min-sum auction I and II where the bid computation is modified as follows: modify step (3) of min-sum auction 
I as rh^ ^^p, = Wij — Taaxi^j{wii — to^~j'^^,} + 5, and modify step (2) of min-sum auction II as bai^pj. — Wij. — Ui + 
S, and hoii^p = Wij — Vi + 5, j ^ ji. For these modified algorithms, we obtain the following result using arguments very 
similar to the ones used in Theorem |5] 

Theorem 7: For S > 0, let cr be the matching obtained from the modified min-sum auction algorithm I (or II). Then, 
Wa > w-K' — n5 (i.e. a is within n5 of the MWM). 

VI. DISCUSSION AND CONCLUSION 

In this paper, we proved that the max-product algorithm converges to the desirable fixed point in the context of finding the 
MWM for a bipartite graph, even in the presence of loops. This result has a twofold impact. First, it will possibly open avenues 
for a demystification of the max-product algorithm. Second, the same approach may provably work for other combinatorial 
optimization problems and possibly lead to better algorithms. 

Using the regularity of the structure of the problem, we managed to simplify the max-product algorithm. In the simplified 
algorithm each node needs to perform 0{n) addition-subtraction operations in each iteration. Since 0{n) iterations are required 
in the worst case, for finite w* and e, the algorithm requires 0{n?) operations at the most. This is comparable with the best 
known MWM algorithm. Furthermore, the distributed nature of the max-product algorithm makes it particularly suitable for 
networking applications like switch scheduling where scalability is a necessary property. 

Future work will consist of trying to extend our result to finding the MWM in a general graph, as our current arguments do 
not carry oveJl Also, we would like to obtain tighter bounds on the running time of the algorithm since simulation studies 
show that the algorithm runs much faster on average than the worst case bound obtained in this paper. 
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key fact in the proof of lemma [3] was the property that bipartite graphs do not have odd cycles. 
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